What's your recovery plan if an entire Qdrant cluster is lost (e.g., all nodes' disks fail) and you only have periodic snapshots?

The recovery plan starts with a clear statement of the recovery point objective (RPO): the amount of data you are willing to lose. With periodic snapshots, the RPO is bounded by the snapshot interval - if you snapshot every hour, you can lose up to an hour of writes. The first step is to identify the most recent successful snapshot for each collection, verify its integrity, and understand what data it contains. The second step is to provision a new cluster with the same or better capacity as the original, with the same collection configuration. The third step is to restore each collection from its snapshot, in parallel if possible, and then re-establish replication and consistency. The fourth step is to replay any writes that occurred after the snapshot, if you have a separate write-ahead log or an upstream source of truth - if you do not, those writes are lost. The fifth step is to validate the restored data: check point counts, run a set of known queries and verify the expected results, and confirm that all indexes and payload schemas are present.

The mechanism of snapshot restore in Qdrant is that a snapshot captures the state of a collection at a point in time, including segments, indexes, and payload. Restoring a snapshot creates a collection with that state. Snapshots are stored as files, either locally or in an object store if configured, and can be downloaded and restored via the API or the CLI. The critical operational details are: snapshots must be stored outside the cluster (in an object store like S3, GCS, or Azure Blob) because if they are only on the cluster's disks, they are lost when the disks fail; the restore process requires a running Qdrant instance to receive the snapshot; and the restore is a collection-level operation, so the cluster must be provisioned and running before the restore can begin. The recovery time objective (RTO) is therefore the sum of: time to provision the cluster, time to download the snapshot, time to restore it, time to rebuild indexes if needed, and time to validate. For large collections, the download and restore can take hours, which is why the RPO and RTO must be planned together and tested.

The trade-off is between snapshot frequency and storage cost. Frequent snapshots reduce RPO but consume more storage and add load to the cluster during snapshot creation. Snapshot creation itself has a cost: it reads the collection's data and writes it to storage, which competes with queries. The right frequency depends on how much data loss is acceptable and how much storage and network bandwidth you have. The common mistake is to store snapshots only on the cluster's own disks, which means they are lost in exactly the scenario you are trying to protect against. The second mistake is to never test the restore procedure, so the first real restore happens during an incident and takes much longer than expected because of unexpected issues. The third mistake is to forget the metadata: the collection configuration, the payload schemas, the aliases, and the shard layout are not part of the snapshot, so you must restore them separately or recreate them from your deployment automation. The fourth mistake is to assume the snapshot is a complete backup - it is a point-in-time copy, and if writes occurred between the snapshot and the failure, those writes are gone unless you have another source. Version note: the snapshot format, the storage options, and the API for creating and restoring snapshots have changed across Qdrant releases. If your recovery plan depends on snapshots, verify the exact commands and their behavior on your version, and test the full restore path end to end.

Version-dependent: the snapshot format and the restore API have changed across Qdrant releases. In some versions, snapshots can be uploaded via the API; in others, they must be placed in a specific directory. The ability to snapshot the entire cluster versus individual collections has also evolved. If your recovery plan depends on snapshots, verify the exact procedure on your version and run a restore drill before you need it in production.